Mobile Device Content Provisioning Adjustments based on Wireless Communication Channel Bandwidth Condition

ABSTRACT

A mobile device includes a display, at least one sensor, and a wireless transceiver. The mobile device also includes control circuitry coupled to the display, the at least one sensor, and the wireless transceiver. The control circuitry is configured to obtain content primitives from the at least one sensor, to perform content provisioning operations to obtain content based at least in part on the content primitives, and to display the obtained content on the display, wherein at least some of the content is virtual content. In response to a bandwidth condition of the wireless communication channel being less than a threshold, the control circuitry is configured to perform adjusted content provisioning operations that involve increasing an amount of image processing operations performed by the mobile device to obtain the content.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to U.S. Provisional PatentApplication No. 62/738,890, filed Sep. 28, 2018.

BACKGROUND

Mobile devices with graphical user interfaces (GUIs) and wirelesscommunication interfaces are widespread. While permanent storage ofcontent to be display on a mobile device is possible, efforts to expandcontent provisioning to mobile devices via wireless communicationchannels are ongoing. There are conflicting limitations related tocameras, display quality, content generation, processing speed, wirelesscommunication channel bandwidth, and system cost that have not beenadequately resolved. Efforts to improve content provisioning to HMDs orother mobile devices via wireless communication channels are ongoing.

SUMMARY

In accordance with at least one embodiment of the disclosure, a mobiledevice comprises a display, at least one sensor, and a wirelesstransceiver. The mobile device also comprises control circuitry coupledto the display, the at least one sensor, and the wireless transceiver.The control circuitry is configured to obtain content primitives fromthe at least one sensor, to perform content provisioning operations toobtain content based at least in part on the content primitives, and todisplay the obtained content on the display, wherein at least some ofthe content is virtual content. In response to a bandwidth condition ofthe wireless communication channel being less than a threshold, thecontrol circuitry is configured to perform adjusted content provisioningoperations that involve increasing an amount of image processingoperations performed by the mobile device to obtain the content.

In accordance with at least one embodiment of the disclosure, a methodcomprises obtaining, by a mobile device, content primitives used togenerate content to be displayed by the mobile device. The method alsocomprises identifying, by the mobile device, a bandwidth condition of awireless communication channel. In response to the bandwidth conditionof the wireless communication channel being less than a threshold, themethod comprises increasing, by the mobile device, an amount of imageprocessing operations performed by the mobile device to generate thecontent. The method also comprises displaying, by the mobile device, thegenerated content, wherein the generated content includescomputer-generated reality (CGR) content.

BRIEF DESCRIPTION OF THE DRAWINGS

For a detailed description of various examples, reference will now bemade to the accompanying drawings in which:

FIG. 1 is a block diagram showing a system in accordance with variousembodiments;

FIG. 2 is a chart showing mobile device features relative to bandwidthconditions of a wireless communication channel in accordance withvarious embodiments;

FIG. 3 is a chart showing a baseline distribution of uplink (mobiledevice to electronic device) bandwidth in accordance with variousembodiments;

FIG. 4 is a chart showing a baseline distribution of downlink(electronic device to mobile device) bandwidth in accordance withvarious embodiments;

FIG. 5 is a chart showing a baseline distribution of uplink (mobiledevice to electronic device) bandwidth demand in accordance with variousembodiments;

FIG. 6 is a chart showing a baseline distribution of downlink(electronic device to mobile device) bandwidth demand in accordance withvarious embodiments;

FIG. 7 is a block diagram showing a system for managing contentprovisioning operations in a mobile device in accordance with variousembodiments;

FIG. 8 is a block diagram of a content provisioning manager for a mobiledevice in accordance with various embodiments;

FIG. 9 is a block diagram showing a computer system configured toperform content provisioning operations in accordance with variousembodiments;

FIG. 10 is a diagram showing downlink (electronic device to mobiledevice) content provisioning adjustments as a function of a channelbandwidth condition in accordance with various embodiments;

FIG. 11 is a block diagram showing a content provisioning manager for anelectronic device in accordance with various embodiments; and

FIG. 12 is a block diagram showing another computer system configured toperform content provisioning operations in accordance with variousembodiments; and

FIG. 13 is a flow chart showing a content provisioning method for amobile device in accordance with various embodiments.

DETAILED DESCRIPTION

Disclosed herein are content display devices, systems, and relatedmethods with content provisioning adjustments based on a wirelesscommunication channel bandwidth condition.

In some embodiments, an example content display system includes a mobiledevice (e.g., a handheld or head-mounted device) and an electronicdevice (e.g., a desktop computer, laptop computer, a base station orother electronic device) in communication via a wireless communicationchannel. Over time, the condition of the wireless communication channelbandwidth varies, which affects the amount of data that can be conveyedfrom the mobile device to the electronic device or vice versa. Inresponse to detected changes in the wireless communication channelbandwidth, content provisioning operations of the mobile device and theelectronic device are updated.

In some embodiments, in response to a bandwidth condition of thewireless communication channel being less than a threshold, the contentprovisioning operations of the mobile device and the electronic deviceare adjusted by increasing an amount of image processing operationsperformed by the mobile device and decreasing an amount of contentconveyed between the electronic device and the mobile device via thewireless communication channel. Thereafter, in response to a bandwidthcondition of the wireless communication channel being greater than athreshold, the content provisioning operations of the mobile device andthe electronic device are adjusted by decreasing an amount of imageprocessing operations performed by the mobile device and increasing anamount of content conveyed between the electronic device and the mobiledevice via the wireless communication channel. Various other contentprovisioning options and related operations are described herein, wheredifferent content provisioning options are used in response to thebandwidth condition of the wireless communication channel (e.g., usingone or more bandwidth condition thresholds and/or an amount of timeabove or below a given bandwidth condition threshold).

Example content provisioning operations performed by the mobile deviceinclude, but are not limited to, priority-based transmission of cameraimages or other content primitives to the electronic device, image/datacompression adjustments for images/data to be conveyed to the electronicdevice via the wireless communication channel, video frames per secondadjustments for camera settings and/or captured camera images, imageresolution adjustments, foveation filter adjustments, and local dataprocessing adjustments (e.g., changing an amount of image/dataprocessing operations and/or the image/data processing algorithms used).As used herein, “content primitives” refers to any image-based data thatcan be used to generate content for display. One example of contentprimitives includes camera image data. In different embodiments, one ormore of the content provisioning operations of the mobile device areadjusted based on bandwidth condition thresholds and/or an amount oftime above or below a given bandwidth condition threshold

Example content provisioning operations performed by the electronicdevice include, but are not limited to, rate control adjustments fordata to be conveyed to the mobile device, foveation filter adjustments,color space adjustments, image/content frames per second adjustments,and downscaling adjustments (e.g., wavelet decomposition adjustments),content generation (e.g., encoded content/video layers), and providingcontent (e.g., encoded content/video layers) to a mobile device. Indifferent embodiments, one or more of the content provisioningoperations of the electronic device are adjusted based on bandwidthcondition thresholds and/or an amount of time above or below a givenbandwidth condition threshold. To provide a better understanding,various content display devices, systems, and methods involving contentprovisioning adjustments performed based on the bandwidth condition of awireless communication channel are described using the figures asfollows.

FIG. 1 is a block diagram showing a content display system 100 inaccordance with various embodiments. As shown, the content displaysystem 100 includes a mobile device 102 (e.g., a handheld orhead-mounted device) and an electronic device 112 in communication witheach other via a wireless communication channel 118. More specifically,the mobile device 102 of FIG. 1 includes a display 104, sensor(s) 106(e.g., cameras or other sensors), a control circuitry 108, and awireless transceiver 110. In some embodiments, the display 104 is aliquid crystal display (LCD) display, an organic light-emitting diode(OLED) display, or a holographic display. Also, the sensor(s) 106 areconfigured to capture ambient environment images. Additionally oralternatively, the sensor(s) 106 may capture images of a user's face(e.g., to track facial expressions), eyes, and/or hands (e.g., to trackhand motions). Additionally or alternatively, the sensor(s) 106 captureimages of a nearby object (e.g., to track object position or motion).The control circuitry 108 of the mobile device 102 performs variousoperations to adjust content provisioning operations based on thebandwidth condition of the wireless communication channel 118. Thecontrol circuitry 108 may also perform other operations. In someembodiments, the control circuitry 108 corresponds to at least oneprocessor and computer-readable medium with software instructions toperform the content provisioning operations described herein.

Example operations of the control circuitry 108 include determining acondition of the wireless communication channel 118, and responding tothe determined condition of the wireless communication channel 118 byadjusting content provisioning operations performed by the mobile device102. In some embodiments, adjustable content provisioning operations ofthe mobile device 102 include, but are not limited to, priority-basedtransmission of content primitives (e.g., camera images or other contentprimitives that can be used to provide content after some processing orother content provisioning operations) to the electronic device 112,image/data compression adjustments for images/data to be conveyed to theelectronic device 112 via the wireless communication channel 118, thevideo frames per second for camera settings and/or captured images,image resolution adjustments, foveation filter adjustments, and localdata processing adjustments (e.g., changing an amount of image/dataprocessing operations and/or the image/data processing algorithms used).In different embodiments, one or more of the content provisioningoperations of the mobile device 102 are adjusted by the controlcircuitry 108 based on bandwidth condition thresholds for the wirelesscommunication channel 118 and/or an amount of time above or below agiven bandwidth condition threshold for the wireless communicationchannel 118.

Meanwhile, the electronic device 112 of FIG. 1 includes a controlcircuitry 114 and a wireless transceiver 116. The control circuitry 114of the electronic device 112 performs various operations to adjustcontent provisioning operations based on the bandwidth condition of thewireless communication channel 118. The control circuitry 114 may alsoperform other operations. In some embodiments, the control circuitry 114corresponds to at least one processor and computer-readable medium withsoftware instructions to perform the content provisioning operationsdescribed herein.

Example operations of the control circuitry 114 include determining acondition of the wireless communication channel 118, and responding tothe determined condition of the wireless communication channel 118 byadjusting content provisioning operations performed by the electronicdevice 112. In some embodiments, adjustable content provisioningoperations of the electronic device 112 include, but are not limited to,rate control adjustments for data to be conveyed to the mobile device102, foveation filter adjustments, color space adjustments,image/content frames per second adjustments, and downscaling adjustments(e.g., wavelet decomposition adjustments), content generation (e.g.,encoded content/video layers), and providing content (e.g., encodedcontent/video layers) to the mobile device 102. In differentembodiments, one or more of the content provisioning operations of theelectronic device 112 are adjusted based on bandwidth conditionthresholds for the wireless communication channel 118 and/or an amountof time above or below a given bandwidth condition threshold for thewireless communication channel 118.

In some embodiments, the operations of the control circuitry 108 and thecontrol circuitry 114 vary depending on the mobile device features inuse. Example mobile device features may be based on cameras including,but are not limited to, eye cameras, hand tracking cameras, facecameras, depth cameras, and scene cameras. Example mobile devicefeatures based on camera images include, but are not limited to,generating content layers, generating 3D objects, video matting, objecttracking (e.g., hand-tracking, eye-tracking, face/expression tracking),and/or other features. In different embodiments, the camera settings(e.g., resolution in ppd or pixels/mm, fps, bit rate) may vary fordifferent features or for different modes of the same mobile devicefeature. When the bandwidth condition of the wireless communicationchannel 118 is above a threshold, more features and related uses ofcameras is possible, where at least some camera image data used isconveyed from the mobile device 102 to the electronic device 112, andrelated content is conveyed from the electronic device 112 to the mobiledevice 102. As the bandwidth condition of the wireless communicationchannel 118 decreases, the set of features or modes being supported canbe adjusted, such that the amount of camera image data conveyed from themobile device 102 to the electronic device 112 via the wirelesscommunication channel 118 is decreased. Similarly, when the set offeatures or modes being supported is adjusted due to a deterioratedbandwidth condition of the wireless communication channel, the amount ofcontent data conveyed from the electronic device 112 to the mobiledevice 102 via the wireless communication channel 118 is decreased. Insuch case, the content provisioning operations of the mobile device 102and the electronic device 112 are adjusted such that the overallfeatures being supported is temporarily decreased, which reduces theamount of data conveyed via the wireless communication channel 118(mobile device to electronic device, or electronic device to mobiledevice).

In one example, the mobile device 102 includes a plurality of sensors106. In response to a bandwidth condition of the wireless communicationchannel 118 being less than a threshold, adjusted content provisioningoperations of the mobile device 102 involve adjusting the operation ofthe sensors 106, adjusting settings of sensor data captured by thesensors 106, and/or omitting conveyance of at some sensor data capturedby the sensors 106 based on a sensor or sensor data prioritizationscheme. In some examples, adjusted content provisioning operations ofthe mobile device 102 involve selectively applying a prioritizationscheme to image processing operations to be performed by the mobiledevice 102 or the electronic device 112 and omitting or migrating someof the image processing operations based on the prioritization scheme.

In one example, in response to a bandwidth condition of the wirelesscommunication channel 118 being less than a threshold, adjusted contentprovisioning operations of the mobile device 102 or the electronicdevice 112 involve selectively adjusting a quantity of image dataprocessing algorithms to be used by the mobile device 102 or theelectronic device 112.

In one example, in response to a bandwidth condition of the wirelesscommunication channel 118 being less than a threshold, adjusted contentprovisioning adjustments of the mobile device 102 involve selectivelyadjusting a compression option for image data captured by the sensor(s)106 and conveyed from the mobile device 102 to the electronic device 112via the wireless communication channel 118.

In one example, in response to a bandwidth condition of the wirelesscommunication channel 118 being less than a threshold, adjusted contentprovisioning operations of the mobile device 102 involve selectivelyadjusting a frames per second for image data captured by the sensor(s)106 and conveyed from the mobile device 102 to the electronic device 112via the wireless communication channel 118.

In one example, in response to a bandwidth condition of the wirelesscommunication channel 118 being less than a threshold, adjusted contentprovisioning operations of the mobile device 102 involve selectivelyadjusting a resolution option for image data by the sensor(s) 106 andconveyed from the mobile device 102 to the electronic device 112 via thewireless communication channel 118.

In one example, in response to a bandwidth condition of the wirelesscommunication channel 118 being less than a threshold, adjusted contentprovisioning operations of the mobile device 102 involve selectivelyadjusting a foveation option for image data captured by the sensor(s)106 and conveyed from the mobile device 102 to the electronic device 112via the wireless communication channel 118.

In one example, the sensor(s) 106 include an eye camera. In response toa bandwidth condition of the wireless communication channel 118 beingless than a threshold, adjusted content provisioning operations of themobile device 102 involve the mobile device 102 selectively performingimage data processing operations associated with the eye camera. In suchcase, eye camera data does not need to be conveyed from the mobiledevice 102 to the electronic device 112, which results in a reducedamount of data being conveyed via the wireless communication channel 118(mobile device to electronic device, or electronic device to mobiledevice).

In one example, the sensor(s) 106 include a hand tracking camera. Inresponse to a bandwidth condition of the wireless communication channelbeing less than a threshold, adjusted content provisioning operations ofthe mobile device 102 involve the mobile device 102 selectivelyperforming image data processing operations associated with the handtracking camera. In such case, hand tracking camera data does not needto be conveyed from the mobile device 102 to the electronic device 112,which results in a reduced amount of data being conveyed via thewireless communication channel 118 (mobile device to electronic device,or electronic device to mobile device).

In one example, the sensor(s) 106 include at least one scene camera andat least one face camera. In response to a bandwidth condition of thewireless communication channel being less than a threshold, adjustedcontent provisioning operations of the mobile device 102 involve themobile device 102 selectively compressing image data associated with theat least one scene camera more than image data associated with the atleast one face camera.

In one example, the sensor(s) 106 include a depth camera, and a scenecamera. In response to a bandwidth condition of the wirelesscommunication channel being less than a threshold, adjusted contentprovisioning operations of the mobile device 102 involve the mobiledevice 102 selectively adjusting a priority of camera image dataassociated with the depth camera and the scene camera. In an exampleprioritization adjustment, some of the data camera image data associatedwith the depth camera and the scene camera is not conveyed from themobile device 102 to the electronic device 112 (or is compression morethan other types of camera image data), which results in a reducedamount of data being conveyed via the wireless communication channel 118(mobile device to electronic device, or electronic device to mobiledevice).

In one example, the sensor(s) 106 include one or more cameras forobtaining head pose tracking data. In response to a bandwidth conditionof the wireless communication channel being less than a threshold,adjusted content provisioning operations of the mobile device 102involve the mobile device 102 selectively processing head pose trackingdata and conveying other camera image data to the electronic device 112.In such case, head pose tracking data does not need to be conveyed fromthe mobile device 102 to the electronic device 112, which results in areduced amount of data being conveyed via the wireless communicationchannel 118 (mobile device to electronic device, or electronic device tomobile device).

In one example, the sensor(s) 106 include one or more cameras forobtaining gaze tracking data. In response to a bandwidth condition ofthe wireless communication channel being less than a threshold, adjustedcontent provisioning operations of the mobile device 102 involve themobile device 102 selectively processing gaze tracking data andconveying other camera image data to the electronic device 112. In suchcase, gaze tracking data does not need to be conveyed from the mobiledevice 102 to the electronic device 112, which results in a reducedamount of data being conveyed via the wireless communication channel 118(mobile device to electronic device, or electronic device to mobiledevice).

In one example, adjusted content provisioning operations of the mobiledevice 102 involve selectively conveying reduced amounts of capturedcamera data for matting operations from the mobile device 102 to theelectronic device 112 until a threshold channel degradation condition ismet, and performing matting operations by the mobile device 102 inresponse to the threshold channel degradation condition being met. Whenthe mobile device 102 performs matting operations, a reduced amount ofdata is conveyed via the wireless communication channel 118 (mobiledevice to electronic device, or electronic device to mobile device).

In one example, adjusted content provisioning operations of the mobiledevice 102 or the electronic device 112 involve eliminating at least oneof 3D room reconstruction operations and object matching operations. Byeliminating 3D room reconstruction operations and object matchingoperations, a reduced amount of data is conveyed via the wirelesscommunication channel 118 (mobile device to electronic device, orelectronic device to mobile device).

In one example, the control circuitry 114 for the electronic device 112and/or the control circuitry 108 for the mobile device 102 is configuredto adjust content provisioning operations that affect the content to bedisplayed on the electronic device 112, where the adjusted contentprovisioning operations of the electronic device 112 involve differentcompression options for content data to be conveyed by the transceiverto the mobile device 102 via the wireless communication channel 118. Forexample, in response to a bandwidth condition of the wirelesscommunication channel 118 being less than a threshold, the controlcircuitry 108 of the mobile device 102 or the control circuitry 114 forthe electronic device 112 adjusts the content provisioning operations ofthe electronic device 112 by selecting between different channeladaptive rate control options. In another example, in response to abandwidth condition of the wireless communication channel 118 being lessthan a threshold, the control circuitry 108 of the mobile device 102 orthe control circuitry 114 for the electronic device 112 adjusts contentprovisioning operations of the electronic device 112 by selectingbetween different MCS states.

In another example, in response to a bandwidth condition of the wirelesscommunication channel 118 being less than a threshold, the controlcircuitry 108 of the mobile device 102 or the control circuitry 114 forthe electronic device 112 adjusts content provisioning operations of theelectronic device 112 by selecting between different foveation filterstrength options. In another example, in response to a bandwidthcondition of the wireless communication channel 118 being less than athreshold, the control circuitry 108 of the mobile device 102 or thecontrol circuitry 114 for the electronic device 112 adjusts contentprovisioning operations of the electronic device 112 by selectingbetween different color space options. In another example, in responseto a bandwidth condition of the wireless communication channel 118 beingless than a threshold, the control circuitry 108 of the mobile device102 or the control circuitry 114 for the electronic device 112 adjustscontent provisioning operations of the electronic device 112 byselecting between different frame per second options. In anotherexample, in response to a bandwidth condition of the wirelesscommunication channel 118 being less than a threshold, the controlcircuitry 108 of the mobile device 102 or the control circuitry 114 forthe electronic device 112 adjusts content provisioning operations of theelectronic device 112 by selecting between different downscalingoptions.

In some embodiments, the control circuitry 114 for the electronic device112 selectively combines a plurality of different compression options inresponse to the bandwidth condition of the wireless communicationchannel 118 being less than a threshold for a predetermined amount oftime, where the plurality of different compression options include datarate options, foveation filter strength options, color space options,frames per second adaptation options, and downscaling options. In oneexample, the control circuitry 114 for the electronic device 112 adjuststhe data rate options as a first priority in response to the bandwidthcondition of the wireless communication channel 118 being less than athreshold for a first time interval. Also, the control circuitry 114 forthe electronic device 112 adjusts the foveation filter strength optionsas a second priority in response to the bandwidth condition of thewireless communication channel 118 being less than a threshold for asecond time interval greater than the first time interval. Also, thecontrol circuitry 114 for the electronic device 112 adjusts the colorspace options as a third priority in response to the bandwidth conditionof the wireless communication channel 118 being less than a thresholdfor a third time interval greater than the second time interval. Also,the control circuitry 114 for the electronic device 112 adjusts theframes per second adaptation options as a fourth priority in response tothe bandwidth condition of the wireless communication channel 118 beingless than a threshold for a fourth time interval greater than the thirdtime interval. Also, the control circuitry 114 for the electronic device112 adjusts the downscaling options as a fifth priority in response tothe bandwidth condition of the wireless communication channel 118 beingless than a threshold for a fifth time interval greater than the fourthtime interval. Other content provisioning prioritization scheme options,including data compression prioritization options for an electronicdevice such as the electronic device 112, are possible. Also, dynamicadjustment of content provisioning prioritization scheme options,including data compression prioritization options for an electronicdevice such as the electronic device 112, are possible. In someexamples, dynamic adjustment of content provisioning prioritizationscheme options are based on channel condition, user preference, contentquality metrics, and/or other criteria. For example, an existingprioritization scheme can be updated to a new prioritization schemebased on the bandwidth condition of a channel, user preference, contentquality metrics, and/or other criteria.

FIG. 2 is a chart 200 showing mobile device features relative tobandwidth conditions of a wireless communication channel such as thewireless communication channel 118 of FIG. 1 in accordance with variousembodiments. In the chart 200 example modulation and coding scheme (MCS)states and related data rates for single channel and channel bondingscenarios are represented. As the MCS state decreases, the availablebandwidth on a wireless communication channel, such as the wirelesscommunication channel 118, decreases. Accordingly, mobile devicefeatures range from “enhanced” near maximum MCS state to “compromised”near minimum MCS state as the available bandwidth for conveyinginformation between an electronic device (e.g., the electronic device112) and a mobile device (e.g., the mobile device 102) becomes morelimited.

As an example, with a maximum MCS state, the mobile device 102 is ableto transfer a large amount of content primitives (e.g., sensor data,camera image data and/or other data) to the electronic device 112. Also,with maximum MCS state, the electronic device 112 uses the contentprimitives to prepare and convey a large amount of content data to themobile device 102. In such case, the content and/or the mobile devicefeatures available for use with the content are enhanced. In contrast,with the minimum MCS state, the mobile device 102 is not able totransfer a large amount of content primitives (e.g., sensor data, cameraimage data and/or other data) to the electronic device 112. Likewise,with the minimum MCS state, the electronic device 112 is unable toconvey a large amount of content data to the mobile device 102. In suchcase, the mobile device 102 prepares more of the content data, but thecontent and/or the mobile device features available for use with thecontent are compromised. Below the minimum MCS state, the mobile device102 transitions to a stand-alone mode, where the mobile device 102generates in own content. In such case, the content and/or the mobiledevice features available for use with the content are compromised. Inthis example, the mobile device 102 is capable of generating content ina stand-alone mode as needed, but the preferred content display scenarioinvolves an exchange of data between the mobile device 102 and theelectronic device 112. Thus, as the MCS state increases, the amount ofdata exchanged between the mobile device 102 and the electronic device112 increases along the quality of content and/or mobile devicefeatures.

FIG. 3 is a chart 300 showing a baseline distribution of uplink (mobiledevice to electronic device) bandwidth in accordance with variousembodiments. In chart 300, the baseline distribution of uplink bandwidthcorresponds to different types of content primitives (e.g., sensor data,camera image data, or other data) provided by a mobile device such asthe mobile device 102. In the example of chart 300, different types ofcontent primitives (CP1-CP7) are represented. In one example, CP1-CP7correspond captured camera image data such as eye camera data, depthcamera data, camera data, and scene camera data. In other example,CP1-CP7 correspond to other types of content primitives. In someembodiments, when the bandwidth condition of a wireless communicationchannel (e.g., the wireless communication channel 118) is above athreshold, the baseline distribution of uplink bandwidth represented inchart 300 is used. Otherwise, when the bandwidth condition of a wirelesscommunication channel (e.g., the wireless communication channel 118) isbelow the threshold, the distribution of uplink bandwidth varies fromwhat is represented in chart 300. For example, the distribution ofuplink bandwidth for certain bandwidth conditions may increase, reduce,or eliminate the bandwidth allocation for one or more types of contentprimitives.

In some examples, mobile device content provisioning options for thebaseline distribution of uplink bandwidth of FIG. 3 involvespredetermined settings for the number of cameras used, image heightoptions (e.g., in pixels), image width options (e.g., in pixels), framesper second options (fps), bits per pixel options (bpp), data rateoptions, and compression options. In response to different bandwidthconditions of a wireless communication channel (e.g., the wirelesscommunication channel 118), the mobile device content provisioningoptions are adjusted from the predetermined settings of a baselinedistribution for uplink bandwidth.

FIG. 4 is a chart 400 showing a baseline distribution of downlink(electronic device to mobile device) bandwidth in accordance withvarious embodiments. In chart 400, the baseline distribution of downlinkbandwidth corresponds to different types of content data provided by anelectronic device (e.g., the electronic device 112). In the example ofchart 400, the different types of content data include matting programdata, secondary display data, layer 0 primary display data, layer 1primary display data, and layer 2 primary display data. In someembodiments, when the bandwidth condition of a wireless communicationchannel (e.g., the wireless communication channel 118) is above athreshold, the baseline distribution of downlink bandwidth representedin chart 500 is used. Otherwise, when the bandwidth condition of awireless communication channel (e.g., the wireless communication channel118) is below the threshold, the distribution of downlink bandwidthvaries from what is represented in chart 500. For example, thedistribution of downlink bandwidth for certain bandwidth conditions mayincrease, reduce, or eliminate the bandwidth allocation for one or moretypes of content primitives.

In some examples, mobile device content provisioning options for thebaseline distribution of downlink bandwidth of FIG. 4 involvespredetermined settings for the number of mobile device displays used,image height options (e.g., in pixels), image width options (e.g., inpixels), frames per second options (fps), bits per pixel options (bpp),data rate options, and compression options. In response to differentbandwidth conditions of a wireless communication channel (e.g., thewireless communication channel 118), the mobile device contentprovisioning options are adjusted from the predetermined settings of abaseline distribution for downlink bandwidth.

FIG. 5 is a chart 500 showing a baseline distribution of uplink (mobiledevice to electronic device) bandwidth demand in accordance with variousembodiments. In chart 500, the baseline distribution of uplink bandwidthdemand corresponds to the uplink bandwidth demand for different types ofcontent primitives (e.g., sensor data, camera image data, and/or otherdata) obtained by a mobile device such as the mobile device 102. Inchart 500, different types of content primitives (CP1-CP7) arerepresented. In one example, CP1-CP7 correspond to captured camera imagedata such as eye camera data, depth camera data, and scene camera data,where the uplink bandwidth demand varies for the different types ofcamera image data. In other example, CP1-CP7 correspond to other typesof content primitives. In some embodiments, when the bandwidth conditionof a wireless communication channel (e.g., the wireless communicationchannel 118) is above a threshold, the baseline distribution of uplinkbandwidth demand represented in chart 500 is used. Otherwise, when thebandwidth condition of a wireless communication channel (e.g., thewireless communication channel 118) is below the threshold, thedistribution of uplink bandwidth demand varies from what is representedin chart 500. For example, the distribution of uplink bandwidth demandfor certain bandwidth conditions may increase, reduce, or eliminate thebandwidth demand allocation for one or more types of content primitives.

In some examples, mobile device content provisioning options for thebaseline distribution of uplink bandwidth demand of FIG. 5 involvespredetermined settings for the number of cameras used, image heightoptions (e.g., in pixels), image width options (e.g., in pixels), framesper second options (fps), bits per pixel options (bpp), data rateoptions, and compression options. In response to different bandwidthconditions of a wireless communication channel (e.g., the wirelesscommunication channel 118), the mobile device content provisioningoptions are adjusted from the predetermined settings of a baselinedistribution for uplink bandwidth demand.

FIG. 6 is a chart 600 showing a baseline distribution of downlink(electronic device to mobile device) bandwidth demand in accordance withvarious embodiments. In chart 600, the baseline distribution of downlinkbandwidth demand corresponds to different types of content data providedby an electronic device (e.g., the electronic device 112). In theexample of chart 600, the different types of content data includematting program data, secondary display data, layer 0 primary displaydata, layer 1 primary display data, and layer 2 primary display data. Insome embodiments, when the bandwidth condition of a wirelesscommunication channel (e.g., the wireless communication channel 118) isabove a threshold, the baseline distribution of downlink bandwidthdemand represented in chart 600 is used. Otherwise, when the bandwidthcondition of a wireless communication channel (e.g., the wirelesscommunication channel 118) is below the threshold, the distribution ofdownlink bandwidth demand varies from what is represented in chart 600.For example, the distribution of downlink bandwidth demand for certainbandwidth conditions may increase, reduce, or eliminate the bandwidthdemand allocation for one or more types of content primitives.

In some examples, mobile device content provisioning options for thebaseline distribution of downlink bandwidth demand of FIG. 6 involvespredetermined settings for the number of mobile device displays used,image height options (e.g., in pixels), image width options (e.g., inpixels), frames per second options (fps), bits per pixel options (bpp),data rate options, and compression options. In response to differentbandwidth conditions of a wireless communication channel (e.g., thewireless communication channel 118), the mobile device contentprovisioning options are adjusted from the predetermined settings of abaseline distribution for downlink bandwidth demand.

FIG. 7 is a block diagram showing a system 700 for managing contentprovisioning operations in a mobile device (e.g., mobile device 102) oran electronic device (e.g., electronic device 112) in accordance withvarious embodiments. The system 700 includes a manager block 702 and anagent block 720. In some embodiments, the manager block 702 with relatedblocks and/or the agent block 720 with related blocks correspond tosoftware instructions executed by one or more processors. As needed,settings for transceiver components and/or other hardware components areupdated based on the operations of the manager block 702 and the agentblock 720.

As shown, the manager block 702 includes a resource manager 704 thatreceives voting results from blocks 706, 708, and 710, where block 706uses active algorithm services to vote, block 708 uses streaming codecstats and rate control states to vote, and block 710 uses channelconditions to vote. The resource manager 704 also receives policyinformation, user preferences, and/or configuration information fromblock 716. Based on the votes from blocks 706, 708, and 710, and/orinformation/preferences from block 716, the response manager 704provides control outputs to blocks 712 and 714, where block 712corresponds to a foveation filter and block 714 corresponds to astreaming codec rate control circuitry 714.

The resource manager 704 is configured to communicate with an agentblock 720. More specifically, the resource manager 704 is configured tosend state sync signals to and to receive state sync signals from aresource agent 722 of the agent block 720. In response to state syncsignals from the resource manager 704 and camera compression stats fromblock 724, the resource agent 722 provides control outputs to a mobiledevice manager 726 and a camera driver 728. In some embodiments, thefoveation filter 712, the streaming codec rate controller 714, themobile device manager 726, and the camera driver 728 adjust availablecontent provisioning options of a mobile device or electronic device,such as those described for the control circuitry 108 and the controlcircuitry 114 in FIG. 1. More specifically, in some embodiments,adjustments to available content provisioning options of a mobile deviceor electronic device, such as those described for the control circuitry108 and the control circuitry 114 in FIG. 1, are based on: 1) votes fromblocks 706, 708, 710; 2) policy/configuration/user preferenceinformation from block 716; and/or 3) compression stats from block 724.

FIG. 8 is a block diagram of a content provisioning manager 802 for amobile device in accordance with various embodiments. As shown, thecontent provisioning manager 802 receives a channel bandwidth condition803 and content provisioning options 804. The channel bandwidthcondition 803 is a current channel bandwidth condition and/or thechannel bandwidth condition for a given time interval. In someembodiments, the content provisioning options 804 include, but are notlimited to, content primitive prioritization options, data/imagecompression options, frames per second options, resolution options,foveation options, and data/image processing algorithm options.

In the example of FIG. 8, content provisioning manager 802 includescontent provisioning migration information 806 that uses the channelbandwidth condition 803 and the content provisioning options 804 toselect content provisioning adjustments, where the adjustments involvetransferring at least some content provisioning operations from theelectronic device to the mobile device or vice versa. Once the contentprovisioning adjustments are selected, the content provisioning manager802 provides control signals 808 to direct components to make theselected content provisioning adjustments.

In some embodiments, the content provisioning manager 802 corresponds tohardware, firmware, and/or software components of a mobile device suchas the mobile device 102 of FIG. 1. Additionally or alternatively, thecontent provisioning manager 802 corresponds to hardware, firmware,and/or software components of an electronic device such as theelectronic device 112 of FIG. 1. In some embodiments, the electronicdevice (e.g., electronic device 112) conveys at least some of thecontrol signals 808 to the mobile device (e.g., mobile device 102) todirect content provisioning adjustments on the mobile device.

FIG. 9 is a block diagram showing a computer system 900 configured toperform content provisioning operations in accordance with variousembodiments. As shown, the computer system 900 includes processor(s) 902in communication with a computer-readable medium 904 (e.g., RAM memory,ROM memory, Flash memory, or combinations thereof). In differentexamples, the processor(s) 902 correspond to one or more centralprocessing units (CPUs), one or more microprocessors, one or moregraphics processors, and/or one or more application-specific processors.

In the example of FIG. 9, the computer-readable medium 904 stores dataor instructions that, when executed by the processor(s) 902, performmobile device content provisioning adjustments based on a channelbandwidth condition as described herein. More specifically, thecomputer-readable medium 904 stores channel condition information 906,which corresponds to instructions, rules, and/or data for determining achannel bandwidth condition. In some embodiments, the channel conditioninformation 906 uses current and/or past channel bandwidth conditionstatistics (e.g., data rate, dropped packets, MCS states, an amount oftime above or below a performance threshold) to determine the channelbandwidth condition.

In the example of FIG. 9, the computer-readable medium 904 also includesvarious content provisioning options in the form of content primitiveinformation 908, compression information 910, frames per second (fps)information 912, resolution information 914, foveation information 916,and algorithm information 918. More specifically, the content primitiveinformation 908 includes instructions, rules, and/or data for selectingcontent primitives (e.g., sensor options, camera options and/or cameraimage settings) in accordance with content provisioning control signalsfor a mobile device such as the control signals 808. Meanwhile, thecompression information 910 includes instructions, rules, and/or datafor selecting data/image compression options in accordance with contentprovisioning control signals for a mobile device such as the controlsignals 808.

Also, fps information 912 includes instructions, rules, and/or data forselecting fps options in accordance with content provisioning controlsignals for a mobile device such as the control signals 808. Theresolution information 914 includes instructions, rules, and/or data forselecting image/content resolution options in accordance with contentprovisioning control signals for a mobile device such as the controlsignals 808. The foveation filter information 916 includes instructions,rules, and/or data for selecting foveation filter options in accordancewith content provisioning control signals for a mobile device such asthe control signals 808. The algorithm information 918 includesinstructions, rules, and/or data for selecting data/image processingalgorithm options in accordance with content provisioning controlsignals for a mobile device such as the control signals 808.

In the example of FIG. 9, the computer-readable medium 904 stores acontent provisioning manager 802A, which is an example of the contentprovisioning manager 802 introduced in FIG. 8. Likewise, the contentprovisioning migration information 806A is an example of the contentprovisioning migration information 806 introduced in FIG. 8. Inoperation, the content provisioning manager 802A provides instructions,rules, and/or data that, when executed by the processor(s) 902, use theavailable channel bandwidth condition and available content provisioningoptions to select content provisioning adjustments. Subsequently,control signal information 924 (e.g., instructions, results, and/ordata) is executed by the processor(s) 902 to determine control signalsto achieve the selected content provisioning adjustments.

In the example of FIG. 9, the computer-readable medium 904 also includesan operating system 920. When executed by the processor(s) 902, theoperating system 920 supports various mobile device functions and/oruser interface functions, including the content provisioning migrationoperations described herein. In some embodiments, the operating system920 includes the content provisioning manager 802A and/or informationused to perform content provisioning adjustments (e.g., information 906,908, 910, 912, 914, 916, 918, and 924).

In one embodiment, the computer system 900 represents components of amobile device such as the mobile device 102 in FIG. 1. In anotherembodiment, the computer system 900 represents components of anelectronic device in a content display system (e.g., the electronicdevice 112 in FIG. 1). In some embodiments, both the mobile device 102and the electronic device 112 include some or all of the componentsrepresented in FIG. 9. As desired, some redundancy of components andoperations can be performed in a mobile device system to achieve thecontent provisioning migration operations described herein whileaccounting for distributed workload options, variable workload scenariosof the electronic device 112, variable workload scenarios of the mobiledevice 102, and/or varying bandwidth conditions of a wirelesscommunication channel.

Once control signals (e.g., control signals 808) for contentprovisioning adjustments are determined (e.g., by the processor(s) 902executing the control signal information 924), these same controlsignals or related control signals are used internally within thecomputer system 900 to perform content provisioning migration.Additionally or alternatively, control signals for content provisioningadjustments are transmitted by the computer system 900 to another device(e.g., electronic device to mobile device, or mobile device toelectronic device) by a wireless transceiver 930 in communication withthe processor(s) 902.

In the example of FIG. 9, the wireless transceiver 930 includes amodulation and coding scheme (MCS) interface 932, which prepares datapackets for transmission via antenna(s) 934 to a wireless communicationchannel such as the wireless communication channel 118 in FIG. 1. Asdesired, the MCS interface 932 can adjust the modulation and/or codingof content and/or content provisioning control signals in accordancewith content provisioning migration operations as described herein.

FIG. 10 is a diagram 1000 showing downlink (electronic device to mobiledevice) content provisioning adjustments as a function of a channelbandwidth condition in accordance with various embodiments. In at leastsome embodiments, the content provisioning adjustments represented inthe diagram 1000 are performed by an electronic device. As shown, thediagram 1000 includes rate control options 1002, foveation filterstrength options 1004, color space options 1006, fps adaptation options1008, and downscaling options 1010. In some embodiments, the options1002, 1004, 1006, 1008, and 1010 are combined depending on the amount oftime spent below a channel bandwidth threshold.

In the example of FIG. 10, the rate control options 1002 include jointsource-channel coding (JSCC) options and/or ProRes codec options. Insome embodiments, content provisioning adjustments based on a channelbandwidth condition involve one or more adjustments to the rate controloptions 1002. In one example, adjustments to the rate control options1002 are initiated when the channel bandwidth condition is less than athreshold level for a first time interval. Thereafter, adjustments tothe rate control options 1002 and/or foveation filter strength options1004 are initiated when the channel bandwidth condition is less than athreshold level for a second time interval that is greater than thefirst time interval. Thereafter, adjustments to the rate control options1002, foveation filter strength options 1004, and/or color space options1006 (e.g., switching between a 444 color space and a 420 color space)are initiated when the channel bandwidth condition is less than athreshold level for a third time interval that is greater than thesecond time interval. Thereafter, adjustments to the rate controloptions 1002, foveation filter strength options 1004, color spaceoptions 1006, and/or fps adaptation options 1008 are initiated when thechannel bandwidth condition is less than a threshold level for a fourthtime interval that is greater than the third time interval. Thereafter,adjustments to the rate control options 1002, foveation filter strengthoptions 1004, color space options 1006, fps adaptation options 1008,and/or downscaling options 1010 (e.g., wavelet options) are initiatedwhen the channel bandwidth condition is less than a threshold level fora fifth time interval that is greater than the fourth time interval. Ifthe channel bandwidth condition improves, the options 1002, 1004, 1006,1008, and 1010 are adjusted to account for the increased bandwidth(e.g., the previous adjustments due to a poor channel bandwidthcondition are reversed at once or reversed over time). In otherembodiments, the order of content provisioning option adjustments variesfrom the example of FIG. 10.

FIG. 11 is a block diagram showing a content provisioning manager 1102for an electronic device in accordance with various embodiments. Asshown, the content provisioning manager 1102 receives a channelbandwidth condition 1103 and content provisioning options 1104. Thechannel bandwidth condition 1103 is a current channel bandwidthcondition and/or the channel bandwidth condition for a given timeinterval. In some embodiments, the content provisioning options 1104include, but are not limited to, rate control options, foveation filteroptions, color space options, fps options, and downscaling options.

In the example of FIG. 11, content provisioning manager 1102 includescontent provisioning migration information 1106 that uses the channelbandwidth condition 1103 and the content provisioning options 1104 toselect content provisioning adjustments, where the adjustments involvetransferring at least some content provisioning operations from theelectronic device to the mobile device or vice versa. Once the contentprovisioning adjustments are selected, the content provisioning manager1102 provides control signals 1108 to direct components to make theselected content provisioning adjustments.

In some embodiments, the content provisioning manager 1102 correspondsto hardware, firmware, and/or software components of an electronicdevice such as the electronic device 112 of FIG. 1. Additionally oralternatively, the content provisioning manager 1102 corresponds tohardware, firmware, and/or software components of a mobile device suchas the mobile device 102 of FIG. 1. In such case, the mobile device(e.g., mobile device 102) conveys at least some of the control signals1108 to the electronic device (e.g., electronic device 112) to directcontent provisioning adjustments on the electronic device.

FIG. 12 is a block diagram showing another computer system 1200configured to perform content provisioning operations in accordance withvarious embodiments. As shown, the computer system 1200 includesprocessor(s) 1202 in communication with a computer-readable medium 1204(e.g., RAM memory, ROM memory, Flash memory, or combinations thereof).In different examples, the processor(s) 1202 correspond to one or morecentral processing units (CPUs), one or more microprocessors, one ormore graphics processors, and/or one or more application-specificprocessors.

In the example of FIG. 12, the computer-readable medium 1204 stores dataor instructions that, when executed by the processor(s) 1202, performelectronic device content provisioning adjustments based on a channelbandwidth condition as described herein. More specifically, thecomputer-readable medium 1204 stores channel condition information 1206,which corresponds to instructions, rules, and/or data for determining achannel bandwidth condition. In some embodiments, the channel conditioninformation 1206 uses current and/or past channel bandwidth conditionstatistics (e.g., data rate, dropped packets, MCS states, an amount oftime above or below a performance threshold) to determine the channelbandwidth condition.

In the example of FIG. 12, the computer-readable medium 1204 alsoincludes various content provisioning options in the form of ratecontrol information 1208, foveation filter information 1210, color spaceinformation 1212, fps adaptation information 1214, and downscalinginformation 1216. More specifically, the rate control information 1208includes instructions, rules, and/or data for selecting rate controlsettings (e.g., JSCC and/or ProRes settings) in accordance with contentprovisioning control signals for an electronic device such as thecontrol signals 1108. Meanwhile, the foveation filter information 1210includes instructions, rules, and/or data for selecting foveation filteroptions in accordance with content provisioning control signals for anelectronic device such as the control signals 1108.

Also, color space information 1212 includes instructions, rules, and/ordata for selecting color space options in accordance with contentprovisioning control signals for an electronic device such as thecontrol signals 1108. The fps adaptation information 1214 includesinstructions, rules, and/or data for selecting fps adaptation options inaccordance with content provisioning control signals for an electronicdevice such as the control signals 1108. The downscaling information1216 includes instructions, rules, and/or data for selecting downscalingoptions in accordance with content provisioning control signals for anelectronic device such as the control signals 1108.

In the example of FIG. 12, the computer-readable medium 1204 stores acontent provisioning manager 1102A, which is an example of the contentprovisioning manager 1102 introduced in FIG. 11. Likewise, the contentprovisioning migration information 1106A is an example of the contentprovisioning migration information 1106 introduced in FIG. 11. Inoperation, the content provisioning manager 1102A provides instructions,rules, and/or data that, when executed by the processor(s) 1202, use thechannel bandwidth condition and available content provisioning optionsto select content provisioning adjustments. Subsequently, control signalinformation 1224 (e.g., instructions, results, and/or data) is executedby the processor(s) 1202 to determine control signals to achieve theselected content provisioning adjustments.

In the example of FIG. 12, the computer-readable medium 1204 alsoincludes an operating system 1220. When executed by the processor(s)1202, the operating system 1220 supports various mobile device functionsand/or user interface functions, including the content provisioningmigration operations described herein. In some embodiments, theoperating system 1220 includes the content provisioning manager 1102Aand/or information used to perform content provisioning adjustments(e.g., information 1206, 1208, 1210, 1212, 1214, and 1216, and 1224).

In one embodiment, the computer system 1200 represents components of anelectronic device such as the electronic device 102 in FIG. 1. Inanother embodiment, the computer system 1200 represents components of amobile device in a content display system (e.g., the mobile device 102in FIG. 1). In some embodiments, both the mobile device 102 and theelectronic device 112 include some or all of the components representedin FIG. 12. As desired, some redundancy of components and operations canbe performed in a content display system to achieve the contentprovisioning migration operations described while accounting fordistributed workload options, variable workload scenarios of theelectronic device 112, variable workload scenarios of the mobile device102, and/or varying bandwidth conditions of a wireless communicationchannel.

Once control signals (e.g., control signals 1108) for contentprovisioning adjustments are determined (e.g., by the processor(s) 1202executing the control signal information 1224), these same controlsignals or related control signals are used internally within thecomputer system 1200 to perform content provisioning migration.Additionally or alternatively, control signals for content provisioningadjustments are transmitted by the computer system 1200 to anotherdevice (e.g., electronic device to mobile device, or mobile device toelectronic device) by a wireless transceiver 1230 in communication withthe processor(s) 1202.

In the example of FIG. 12, the wireless transceiver 1230 includes an MCSinterface 1232, which prepares data packets for transmission viaantenna(s) 1234 to a wireless communication channel such as thecommunication channels 118 in FIG. 1. As desired, the MCS interface 1232can adjust the modulation and/or coding of content and/or contentprovisioning control signals in accordance with content provisioningmigration operations as described herein.

FIG. 13 is a flow chart showing a content provisioning method 1300 for acontent display system in accordance with various embodiments. As shown,the method 1300 comprises obtaining content primitives (e.g., cameraimages) used to generate content to be displayed by a mobile device atblock 1302. At block 1304, a bandwidth condition of a wirelesscommunication channel (e.g., the wireless communication channel betweenthe mobile device and an electronic device) is determined. At block1306, content provisioning operations of the electronic device and/orthe mobile device are adjusted based on the determined bandwidthcondition of the wireless communication channel.

In some examples, the content provisioning method 1300 involves mobiledevice operations at block 1302 including obtaining, by a mobile device,content primitives used to generate content to be displayed by themobile device. Also, the mobile device operations include identifying,by the mobile device, a bandwidth condition of a wireless communicationchannel at block 1304. In response to the bandwidth condition of thewireless communication channel being less than a threshold, block 1306may include the mobile device increasing an amount of image processingoperations performed by the mobile device to generate the content. Oncethe content is generated, the mobile device is able to display thegenerated content. In some examples, the generated content includescomputer-generated reality (CGR) (e.g., augmented reality, mixedreality, virtual reality, and the like) content.

In some examples, the content provisioning method 1300 includesreceiving, by an electronic device, content primitives via a wirelesscommunication channel. The method 1300 also includes performing, by theelectronic device, content provisioning operations based on the receivedcontent primitives, wherein the content provisioning operations involvegenerating content image data and transmitting the content image data tothe wireless communication channel, wherein at least some the contentimage data corresponds to CGR content. In response to the bandwidthcondition of the wireless communication channel being less than athreshold, the method 1300 also includes adjusting the contentprovisioning operations by decreasing an amount of content image dataconveyed to the wireless communication channel.

In one example, in response to the bandwidth condition of the wirelesscommunication channel being less than a threshold, the operations ofblock 1306 involve adjusting content provisioning operations of theelectronic device by decreasing an amount of content data conveyed bythe electronic device to the mobile device via the wirelesscommunication channel. Subsequently, if the bandwidth condition of thewireless communication channel improves (e.g., the bandwidth conditionis greater than the threshold), the operations of block 1306 involveadjusting content provisioning operations of the electronic device byincreasing an amount of content data conveyed by the electronic deviceto the mobile device via the wireless communication channel.

In some embodiments, the operations of block 1306 involve decreasing orincreasing the amount of content data to be conveyed from an electronicdevice to a mobile device via the wireless communication channel basedon different compression options. Example compression options includedifferent channel adaptive rate control options, different MCS states,different foveation filter strength options, different color spaceoptions, different frame per second options, and/or differentdownscaling options. As desired, the use and adjustment of differentcompression options are selected based on one or more bandwidthcondition thresholds such as MCS state, available data rate and/orpredetermined time intervals at an MCS state or data rate.

In some embodiments, the operations of block 1306 involve aprioritization scheme to select adjustments. As an example, in someembodiments, adjustments to data rate options, foveation filter strengthoptions, color space options, frames per second adaptation options, anddownscaling options are selected based on a predetermined prioritizationscheme. In one example, adjusting data rate options is a first priorityin response to the bandwidth condition of the wireless communicationchannel being less than a threshold for a first time interval. Also,adjusting foveation filter rate options is a second priority in responseto the bandwidth condition of the wireless communication channel beingless than a threshold for a second time interval greater than the firsttime interval. Also, adjusting the color space options is a thirdpriority in response to the bandwidth condition of the wirelesscommunication channel being less than a threshold for a third timeinterval greater than the second time interval. Also, frames per secondadaptation options is a fourth priority in response to the bandwidthcondition of the wireless communication channel being less than athreshold for a fourth time interval greater than the third timeinterval. Also, adjusting the downscaling options is a fifth priority inresponse to the bandwidth condition of the wireless communicationchannel being less than a threshold for a fifth time interval greaterthan the fourth time interval. In some embodiments, the prioritizationscheme is dynamic and adjustable based on user preferences, contentquality metrics, and/or other criteria.

In some embodiments, the operations of block 1306 involve decreasing orincreasing the amount of content data to be conveyed via the wirelesscommunication channel by increasing an amount of image processingoperations performed by the mobile device and decreasing an amount ofcontent conveyed by the electronic device to the mobile device via thewireless communication channel.

Example content provisioning adjustments by a mobile device include, butare not limited to, decreasing an amount of content primitives (e.g.,sensor data or camera image data) conveyed from the mobile device to theelectronic device, omitting at least some image processing operationsperformed by at least one of the mobile device and the electronicdevice, adjusting a quantity of image data processing algorithms to beused by the mobile device or the electronic device, adjusting acompression option for captured image data to be conveyed from themobile device to the electronic device via the wireless communicationchannel, adjusting a frames per second for captured image data to beconveyed from the mobile device to the electronic device via thewireless communication channel, adjusting a resolution option forcaptured image data to be conveyed from the mobile device to theelectronic device via the wireless communication channel, and/oradjusting a foveation option for captured image data to be conveyed fromthe mobile device to the electronic device via the wirelesscommunication channel.

In some embodiments, the operations of block 1306 involve responding toa determination that the wireless communication channel stays less athreshold for multiple time intervals. In response to the bandwidthcondition of the wireless communication channel staying less a thresholdfor multiple time intervals, the amount of content generation operationsperformed by the mobile device increases for each time interval and theamount of content conveyed by the electronic device to the mobile devicedecreases for each time interval.

In some embodiments, the operations of block 1306 involve switchingimage data processing operations associated with an eye camera from theelectronic device to the mobile device, switching image data processingoperations associated with a hand tracking camera from the electronicdevice to the mobile device, compressing camera image data associatedwith the at least one scene camera more than camera image dataassociated with the at least one face camera, adjusting a priority ofcamera image data associated with a depth camera and a scene camera ofthe mobile device, switching processing of head pose tracking data fromthe electronic device to the mobile device, and/or switching processingof gaze tracking data from the electronic device to the mobile device.

In some embodiments, the operations of block 1306 involve conveyingreduced amounts of content primitives (e.g., camera image data) formatting operations from the mobile device to the electronic device untila threshold channel degradation condition is met, and performing mattingoperations by the mobile device in response to the threshold channeldegradation condition being met. In some embodiments, the operations ofblock 1306 involve eliminating at least some 3D reconstructionoperations and/or at least some object matching operations performed bythe mobile device or the electronic device.

In some embodiments, the content provision operations of the mobiledevice and/or the electronic device described herein are employedindividually (e.g., only one mobile device adjustment is made, only oneelectronic device adjustment is made, etc.). In other embodiments, thecontent provision operations of the mobile device and/or the electronicdevice described herein are combined in different ways to achieve adesired content quality metric for a given bandwidth condition. Fordifferent embodiments, the workload capabilities of the mobile deviceand/or the electronic device vary, which affects how content provisionoperations are performed and adjusted in response to the bandwidthcondition of a wireless communication channel.

While much of the discussion has been directed to adjusting contentprovisioning operations of a mobile device and/or electronic device toreduce the amount of content data conveyed via a wireless communicationchannel in response to a compromised bandwidth condition, it should beappreciated that the bandwidth condition is dynamic. Accordingly,content provisioning adjustments of a mobile device and/or electronicdevice in response to improvements to the bandwidth condition of awireless communication channel may also be performed. In someembodiments, control of content provisioning adjustment operations of amobile device and/or electronic device in response to a dynamicbandwidth condition of a wireless communication channel may beaccomplished using a default set of content provisioning operations foreach of a plurality of bandwidth conditions. Additionally oralternatively, adjustments to content provisioning operations fordifferent bandwidth conditions of a wireless communication channel ispossible based on user preferences, content quality metrics, and/orother criteria.

Certain terms have been used throughout this description and claims torefer to particular system components. As one skilled in the art willappreciate, different parties may refer to a component by differentnames. This document does not intend to distinguish between componentsthat differ in name but not function. In this disclosure and claims, theterms “including” and “comprising” are used in an open-ended fashion,and thus should be interpreted to mean “including, but not limited to .. . .” Also, the term “couple” or “couples” is intended to mean eitheran indirect or direct wired or wireless connection. Thus, if a firstdevice couples to a second device, that connection may be through adirect connection or through an indirect connection via other devicesand connections. The recitation “based on” is intended to mean “based atleast in part on.” Therefore, if X is based on Y, X may be a function ofY and any number of other factors.

The above discussion is meant to be illustrative of the principles andvarious embodiments of the present invention. Numerous variations andmodifications will become apparent to those skilled in the art once theabove disclosure is fully appreciated. It is intended that the followingclaims be interpreted to embrace all such variations and modifications.

What is claimed is:
 1. A mobile device, comprising: a display; at leastone sensor; a wireless transceiver; and control circuitry coupled to thedisplay, the at least one sensor, and the wireless transceiver, whereinthe control circuitry is configured to obtain content primitives fromthe at least one sensor, to perform content provisioning operations toobtain content based at least in part on the content primitives, and todisplay the obtained content on the display, wherein at least some ofthe displayed content is virtual content, and wherein, in response to abandwidth condition of the wireless communication channel being lessthan a threshold, the control circuitry is configured to performadjusted content provisioning operations that involve increasing anamount of image processing operations performed by the mobile device toobtain the content.
 2. The mobile device of claim 1, wherein the atleast one sensor comprises a plurality of cameras configured to provideimage data as content primitives, and wherein the adjusted contentprovisioning operations involve decreasing an amount of contentprimitive data conveyed by the wireless transceiver to the wirelesscommunication channel.
 3. The mobile device of claim 1, wherein theadjusted content provisioning operations comprise applying aprioritization of image processing operations to be performed by themobile device and omitting some of the image processing operations basedon the prioritization.
 4. The mobile device of claim 1, wherein theadjusted content provisioning operations comprise increasing a quantityof image data processing algorithms to be used by the mobile device. 5.The mobile device of claim 1, wherein the adjusted content provisioningoperations comprise applying a compression option for captured imagedata to be conveyed by the wireless transceiver to the wirelesscommunication channel.
 6. The mobile device of claim 1, wherein theadjusted content provisioning comprise decreasing a frames per secondfor captured image data to be conveyed by the wireless transceiver tothe wireless communication channel.
 7. The mobile device of claim 1,wherein the adjusted content provisioning operations comprise decreasinga resolution option for captured image data to be conveyed by thewireless transceiver to the wireless communication channel.
 8. Themobile device of claim 1, wherein the adjusted content provisioningoperations comprise adjusting a foveation option for captured image datato be conveyed by the wireless transceiver to the wireless communicationchannel.
 9. The mobile device of claim 1, wherein the at least onesensor comprises a camera, and wherein the adjusted content provisioningoperations comprise performing image data processing operationsassociated with the camera.
 10. The mobile device of claim 1, whereinthe at least one sensor comprises a first camera and a second camera,and wherein the adjusted content provisioning operations comprisecompressing image data associated with the first camera more than imagedata associated with the second camera.
 11. The mobile device of claim1, wherein the at least one sensor comprises a first camera and a secondcamera, and wherein the adjusted content provisioning operationscomprise processing image data of the first camera and conveying imagedata of the second camera to the wireless communication channel.
 12. Themobile device of claim 1, wherein the at least one sensor comprises aplurality of cameras, and wherein the adjusted content provisioningoperations comprise conveying reduced amounts of captured camera datafor matting operations to the wireless communication channel until athreshold channel degradation condition is met, and performing mattingoperations by the mobile device in response to the threshold channeldegradation condition being met.
 13. The mobile device of claim 1,wherein the adjusted content provisioning operations compriseeliminating at least one of 3D room reconstruction operations and objectmatching operations.
 14. A method, comprising: obtaining, by a mobiledevice, content primitives used to generate content to be displayed bythe mobile device; identifying, by the mobile device, a bandwidthcondition of a wireless communication channel; in response to thebandwidth condition of the wireless communication channel being lessthan a threshold, increasing, by the mobile device, an amount of imageprocessing operations performed by the mobile device to generate thecontent; and displaying, by the mobile device, the generated content,wherein the generated content includes computer-generated reality (CGR)content.
 15. The method of claim 14, further comprising, in response tothe bandwidth condition of the wireless communication channel being lessthan a threshold, decreasing an amount of content primitives conveyedfrom the mobile device to the wireless communication channel.
 16. Themethod of claim 14, further comprising, in response to the bandwidthcondition of the wireless communication channel being less than athreshold, omitting at least some image processing operations performedby the mobile device.
 17. The method of claim 14, further comprising, inresponse to the bandwidth condition of the wireless communicationchannel being less than a threshold, adjusting a compression option forcaptured image data to be conveyed from the mobile device to thewireless communication channel.
 18. The method of claim 14, furthercomprising, in response to the bandwidth condition of the wirelesscommunication channel being less than a threshold, adjusting a framesper second, a resolution option, and a foveation option for capturedimage data to be conveyed from the mobile device to the wirelesscommunication channel.
 19. The method of claim 14, further comprising,conveying reduced amounts of camera image data for matting operationsfrom the mobile device to the wireless communication channel until athreshold channel degradation condition is met, and performing mattingoperations by the mobile device in response to the threshold channeldegradation condition being met.
 20. The method claim 14, furthercomprising, in response to the bandwidth condition of the wirelesscommunication channel being less than a threshold, eliminating at leastsome 3D reconstruction operations or at least some object matchingoperations performed by the mobile device.